Understanding multimodal deixis with gaze and gesture in conversational interfaces
نویسنده
چکیده
منابع مشابه
Eye Gaze for Reference Resolution in Multimodal Conversational Interfaces
EYE GAZE FOR REFERENCE RESOLUTION IN MULTIMODAL CONVERSATIONAL INTERFACES
متن کاملAn Exploration of Eye Gaze in Spoken Language Processing for Multimodal Conversational Interfaces
Motivated by psycholinguistic findings, we are currently investigating the role of eye gaze in spoken language understanding for multimodal conversational systems. Our assumption is that, during human machine conversation, a user’s eye gaze on the graphical display indicates salient entities on which the user’s attention is focused. The specific domain information about the salient entities is ...
متن کاملMulti-Modal Conversational Search and Browse
In this paper, we create an open-domain conversational system by combining the power of internet browser interfaces with multi-modal inputs and data mined from web search and browser logs. The work focuses on two novel components: (1) dynamic contextual adaptation of speech recognition and understanding models using visual context, and (2) fusion of users’ speech and gesture inputs to understan...
متن کاملEye Gaze for Attention Prediction in Multimodal Human-Machine Conversation
In a conversational system, determining a user’s focus of attention is crucial to the success of the system. Motivated by previous psycholinguistic findings, we are currently examining how eye gaze contributes to automated identification of user attention during humanmachine conversation. As part of this effort, we investigate the contributing roles of various features that are extracted from e...
متن کاملWolfgang Wahlster
In face-to-face conversation humans frequently use deictic gestures parallel to verbal descriptions for referent identification. Such a multimodal form of communication is of great importance for intelligent interfaces, because it simplifies and speeds up reference to objects in a visual context. Natural pointing behavior is very flexible, but possibly ambiguous or vague, so that without a care...
متن کامل